Maximum Roaming Multi-Task Learning

Multi-task learning has gained popularity due to the advantages it provides with respect resource usage and performance. Nonetheless, joint optimization of parameters multiple tasks remains an active research topic. Sub-partitioning between different proven be efficient way relax constraints over shared weights, may partitions disjoint or overlapping. However, one drawback this approach is that can weaken inductive bias generally set up by task optimization. In work, we present a novel partition parameter space without weakening bias. Specifically, propose Maximum Roaming, method inspired dropout randomly varies partitioning, while forcing them visit as many possible at regulated frequency, so network fully adapts each update. We study properties our through experiments on variety visual multi-task data sets. Experimental results suggest regularization brought roaming more impact performance than usual partitioning strategies. The overall flexible, easily applicable, superior consistently achieves improved performances compared recent formulations.